# ChartsMaze EDL Pipeline

## Docs

- [Advanced Metrics Processor](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/advanced-metrics-processor.md): Processes OHLCV data to calculate advanced volume, volatility, and price metrics
- [Bulk Market Analyzer](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/bulk-market-analyzer.md): Core fundamental and technical analysis script that generates comprehensive stock analysis data
- [Corporate Events Processor](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/corporate-events.md): Maps corporate actions, surveillance status, deals, and news headlines to stock records
- [Earnings Performance Processor](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/earnings-performance.md): Calculates stock performance metrics relative to earnings announcement dates
- [enrich_fno_data.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/enrich-fno-data.md): F&O Data Enrichment - Adds F&O flags, lot sizes, and next expiry dates to the master stock database
- [fetch_advanced_indicators.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-advanced-indicators.md): Fetches technical indicators including pivot points, EMA, SMA, and RSI signals
- [fetch_bulk_block_deals.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-bulk-block-deals.md): Fetches bulk and block deal transactions from the last 30 days using Dhan Static ScanX API
- [fetch_circuit_stocks.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-circuit-stocks.md): Fetches stocks hitting upper or lower circuit limits from Dhan ScanX Analytics API
- [fetch_company_filings.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-company-filings.md): Fetches regulatory filings from dual endpoints with deduplication
- [fetch_complete_price_bands.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-complete-price-bands.md): Fetches the complete daily securities list with price bands from NSE Archives
- [fetch_corporate_actions.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-corporate-actions.md): Fetches historical and upcoming corporate actions (dividends, bonuses, splits)
- [fetch_dhan_data.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-dhan-data.md): Fetches full market data for all NSE stocks from Dhan ScanX API
- [fetch_etf_data.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-etf-data.md): Standalone script to fetch NSE ETF (Exchange Traded Funds) data
- [fetch_fno_data.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-fno-data.md): Standalone script to fetch NSE F&O (Futures & Options) stocks data
- [fetch_fundamental_data.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-fundamental-data.md): Fetches quarterly financial results and valuation ratios for all stocks
- [fetch_incremental_price_bands.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-incremental-price-bands.md): Fetches daily price band changes from NSE Archives CSV files
- [fetch_all_indices.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-indices.md): Standalone script to fetch NSE indices data with live OHLC and volume
- [fetch_indices_ohlcv.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-indices-ohlcv.md): Ultra-fast Index OHLCV Fetcher - Hybrid incremental data sync for major Indian indices
- [fetch_market_news.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-market-news.md): Fetches AI-sentiment analyzed news feed for each stock
- [fetch_new_announcements.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-new-announcements.md): Fetches live corporate announcements for all stocks from the Dhan announcements API
- [fetch_all_ohlcv.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-ohlcv.md): Fetches historical OHLCV (Open, High, Low, Close, Volume) data with incremental updates
- [fetch_surveillance_lists.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/fetch-surveillance-lists.md): Fetches NSE Additional Surveillance Measure (ASM) and Graded Surveillance Measure (GSM) lists
- [pipeline_utils.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/pipeline-utils.md): Shared utilities module providing common constants and helper functions
- [process_historical_market_breadth.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/process-historical-market-breadth.md): Historical Market Breadth Calculator - Generates time-series breadth metrics for charting and trend analysis
- [process_market_breadth.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/process-market-breadth.md): Market Breadth Analytics Engine - Calculates RS scores and generates sector/industry breadth metrics
- [single_stock_analyzer.py](https://mintlify.wiki/Anny26022/chartsmaze_clone/api/single-stock-analyzer.md): Utility script for deep fundamental analysis of individual stocks
- [Data Flow](https://mintlify.wiki/Anny26022/chartsmaze_clone/concepts/data-flow.md): How data transforms and flows through the 6-phase EDL Pipeline
- [Output Schema](https://mintlify.wiki/Anny26022/chartsmaze_clone/concepts/output-schema.md): Complete reference for the 86 fields in all_stocks_fundamental_analysis.json
- [Pipeline Architecture](https://mintlify.wiki/Anny26022/chartsmaze_clone/concepts/pipeline-architecture.md): Understanding the 6-phase design and dependency structure of the EDL Pipeline
- [GitHub Actions Workflow](https://mintlify.wiki/Anny26022/chartsmaze_clone/configuration/github-actions.md): Automated daily data refresh using GitHub Actions with intelligent OHLCV caching
- [Pipeline Settings](https://mintlify.wiki/Anny26022/chartsmaze_clone/configuration/pipeline-settings.md): Configure the EDL pipeline execution behavior with three key flags
- [Dhan API Endpoints](https://mintlify.wiki/Anny26022/chartsmaze_clone/data/dhan-endpoints.md): Complete reference for all Dhan ScanX API endpoints used in the EDL Pipeline
- [Event Markers](https://mintlify.wiki/Anny26022/chartsmaze_clone/data/event-markers.md): Visual event indicators for corporate actions, surveillance, and regulatory events
- [Field Reference](https://mintlify.wiki/Anny26022/chartsmaze_clone/data/field-reference.md): Complete reference for all 86 fields in the output JSON
- [NSE Endpoints](https://mintlify.wiki/Anny26022/chartsmaze_clone/data/nse-endpoints.md): Direct NSE India endpoints for price bands, equity lists, and market data
- [Output Files](https://mintlify.wiki/Anny26022/chartsmaze_clone/data/output-files.md): Complete reference for all files generated by the EDL Pipeline
- [Rate Limits & Concurrency](https://mintlify.wiki/Anny26022/chartsmaze_clone/data/rate-limits.md): Threading configuration, timeout values, and best practices to avoid rate limiting
- [Installation](https://mintlify.wiki/Anny26022/chartsmaze_clone/installation.md): Set up the EDL Pipeline environment with Python and dependencies
- [ChartsMaze EDL Pipeline](https://mintlify.wiki/Anny26022/chartsmaze_clone/introduction.md): Enterprise Data Lake for NSE Stock Market - Complete fundamental, technical, and event data for 2,775 stocks
- [Phase 1: Core Data](https://mintlify.wiki/Anny26022/chartsmaze_clone/pipeline/phase-1-core-data.md): Foundation layer that fetches master stock list and fundamental data
- [Phase 2: Data Enrichment](https://mintlify.wiki/Anny26022/chartsmaze_clone/pipeline/phase-2-enrichment.md): Fetches supplementary data: filings, news, corporate actions, and market indicators
- [Phase 3: Base Analysis](https://mintlify.wiki/Anny26022/chartsmaze_clone/pipeline/phase-3-analysis.md): Builds the master JSON structure by merging all fetched data
- [Phase 4: Enrichment Injection](https://mintlify.wiki/Anny26022/chartsmaze_clone/pipeline/phase-4-injection.md): Modifies master JSON in-place to inject advanced metrics, F&O data, and event markers
- [Quick Start](https://mintlify.wiki/Anny26022/chartsmaze_clone/quickstart.md): Run your first EDL Pipeline and produce complete NSE stock market data in minutes

## OpenAPI Specs

- [openapi](https://mintlify.wiki/Anny26022/chartsmaze_clone/api-reference/openapi.json)